Dudley Castle is a ruined fortification and a Grade I listed building. It is thought that one of William the Conqueror's followers, Ansculf de Picquigny, built the first castle around 1070. His son, William Fitz-Ansculf, was in possession of the castle when it was recorded at the time of the Domesday Book of 1086. The castle is under the control of Dudley Zoo which occupies the grounds of the castle.

 

The castle is reputedly haunted by several ghosts. A "mysterious figure" has been sighted on the parapets; the ghost of an old lady haunts the ramparts (apparently legend has it that she committed suicide there when her cat was killed by local youths on a date yet to be ascertained, but whilst the castle was inhabited); several hooded figures sighted together; a drummer believed to be from the English civil war; and the Grey Lady, believed to be Dorothy Beaumont who died during the castle seige of 1646.